Neural Networks: Computational Models and Applications - ReadingSample

1 Introduction Typically, models of neural networks are divided into two categories in terms of signal transmission manner: feed-forward neural networks and recurrent neural networks. They are built up using different frameworks, which give rise to different fields of applications. Feed-forward neural network (FNN), also referred to as multilayer percep-trons (MLPs), has drawn great interests over the last two decades for its distinction as a universal function approximator As an important intelligent computation method, FNN has been applied to a wide range of applications, including curve fitting, pattern classification and nonlinear system identification and so on (Vemuri, 1995). FNN features a supervised training with a highly popular algorithm known as the error back-propagation algorithm. In the standard back-propagation (SBP) algorithm, the learning of a FNN is composed of two passes: in the forward pass, the input signal propagates through the network in a forward direction, on a layer-by-layer basis with the weights fixed; in the backward pass, the error signal is propagated in a backward manner. The weights are adjusted based on an error-correction rule. Although it has been successfully used in many real world applications, SBP suffers from two infamous shortcomings, i.e., slow learning speed and sensitivity to parameters. Many iterations are required to train small networks, even for a simple problem. The sensitivity to learning parameters, initial states and perturbations was analyzed in (Yeung and Sun, 2002). Behind such drawbacks the learning rate plays a key role in affecting the learning performance and it has to be chosen carefully. If the learning rate is large, the network may exhibit chaotic